Some devices that access stored data (e.g., a display controller that accesses data within a computing device) may have a low tolerance for latency associated with access of the data from a main memory. Consequently, such a device may require that a significant proportion of a cache memory be allocated for use by the device.
Due at least in part to virtualization techniques and operating system behavior, the main memory allocated to the device may not be contiguous, but may be formed from tens of thousands, or even hundreds of thousands of small memory pages, each at an arbitrary location in main memory. Due to the low latency requirements of the device that accesses the data, virtually all of these memory pages may need to be locked into the cache.
In order to be able to lock in every statistically plausible combination of memory page addresses, cache associativity may be many times higher than an average number of ways used by the device. For example, if the cache is 16-way set associative and 50% of the cache memory is assigned to arbitrary pages allocated to a device, the average number of ways assigned to the device will be 8. Yet for some sets, greater than the 16 total available ways will be required.